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ABSTRACT 

A large-scale study was designed to assess the extent 
of emotional disturbance among Head Start children and to provide a 
consistent basis for selection if therapeutic intervention were 
indicated. The study* s aim was to avoid the problem of shifting 
baselines by individual teachers for determining the degree to which 
their children were departing from normalcy and the tolerance limits 
they were willing to accept before assigning a child to therapeutic 
treatment. A total of 413 children were tested using Kohn's Behavior 
Checklist and Competence Scale to assess their overt functioning in 
-group settings. The study seems to support the usefulness of this 
instrument and procedure for identifying, at an early age, children 
who are later likely to have severe problems. The question of whether 
therapeutic intervention with this population can have an appreciable 
impact on changing this prediction was the subject of a subsequent 
study. An examiner's manual for the Behavior Checklist and Competence 
Scale is included in this report. (LH) 
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Problem 

Of all the components of comprehensive child care specified in the Head 
Start guidelines, mental health and psychological services have been most 
frequently neglected. This is true in spite of the fact that the Joint Com- 
mission on Mental Health of Children (1970), reporting only on the school- 
age population, noted that close to a million and a half children in the 
United States were suffering from severe mental problems. In addition, 10 
to 12 percent of those enrolled in the elementary school had demonstrated 
intermittent but persistent emotional disturbances. Although comparable 
statistics for preschool children are extremely difficult to obtain, there is 
certainly no rational or scientific basis to assume that young children, es- 
pecially those from disadvantaged, poverty families, do not have their fair 
share of mental problems. 

Neglecting the needs in this important health area is exceedingly costly. 
According to the report of the President's Task Force (1970), expenditures for 
mental health care alone are over 20 billion dollars a year. This does not 
include the financial burden placed on the courts and criminal institutions 
by destructive aberrant behavior, nor does it in any way count the cost in 
loss of productivity, let alone human lives. This situation is particularly 
tragic since there is general agreement, even among widely disparate theoretical 
orientations, that the etiology of mental illness is rooted in the early years 
of childhood. More tragic still is the fact that, whereas deviant behavior 
patterns of adolescents and adults are extremely difficult to modify, young 
children respond quite readily to appropriate intervention procedures. 

While there are many reasons why this important component of health care 
for Head Start children has received scant attention, a major factor is the 
insufficient number of available personnel with traditional professional 
qualifications. Several innovative programs have been designed to provide 
alternative approaches to serving the needs of these children. A program, for 
the training of preschool teachers to work with emotionally disturbed children 
was established by the National Institutes of Mental Health in 1965, with 
chants awarded to Tufts University, Wheelock College, University of Michigan, 
and Cedar-Sinai Medical Center in Los Angeles. Since his appointment as 
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Director of Psychological Services in the Office of Child Development, Dr. 

Paul Wohlford (cf. Wohlford, 1972) has attempted to focus attention on this 
critical area, and to upgraoe services by integrating anc coordinating other 
public and private agencies with capacity to augment the resources of 
budgetarily limited Head Start programs. 

Recently Stern, Nummedal , & Brussel 1 (1972) completed a research study 
comparing the effectiveness of three procedures for working with disturbed 
Head Start children. The unevenness and unreliability of teacher identifi- 
cation of eligible children for this study pointed up the need for a selec- 
tion instrument- which would provide an objective and reliable method for 

describing the significant symptomology. 

The lack of measures for use with preschool children is a presisting 
problem in early childhood education. Difficult as It may be to design tests 
of cognitive functioning for use at this age, a far more challenging area of 
assessment is that concerned with affective behaviors. Particularly is there 
a dearth of non-projective diagnostic devices on which to base psychological 
and educational decisions. Fortunately, an early version of the Kohn (1968) 
Behavior Checklist^ and Competence Scale was made available to the project. 

The literature on which the theoretical rationale is based and the details 
of the development of the final measure are presented in a recent article 
(Kohn & Rosman, 1972a). For the convenience of the reader of the present study, 
it should be noted that the most relevant references cited include Baumrind 
(1971), Biber (1958), Chance (1959), Digman (1963), Jahoda (1958), Lorr, Klett 
& McNair (1963), Peterson (1961 and 1963), Schaefer (1961), Schaefer & Aaronson 
(1966), and Walker (1967). 

The factor structure of the two associated scales was examined in terms 
of congruence with factors on similar measures (e.g. those of Schaefer and 
Peterson), and there is substantial basis for accepting the generality of 
the constructs. Further exploratory work’ with the Kohn instrument was con- 
cerned with the persistence of the factor structures over time, both within 
cay care and from day care to elementary school. The authors conclude that: 
'The predictions. . .about the persistence of personality trends are well 
supported by the findings..." (p.441). 



^ In his work, Kohn has interchangeably 
'Behavior 11 tc describe the Checklist; 



used "Problem," "Symptom," and 

the latter term is used in this paper. 
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A second study by the same authors (Kohn & Rosman, 1972b) provides 
evidence of the usefulness of the instrument for relating social-emotional 
functioning in the preschool to later intellectual achievement. The con- 
gruence, stability, and predictive power of the measure thus justified its 
adoption as the screening device for the Stern, et al. study. 



Description of the Instrument : 

Kohn's Behavior Checklist and Competence Scale was designed to assess the 
overt functioning of children in group settings. The first part, the Behavior 
Checklist, includes items which are generally recognized as clinical symptoms 
of disturbance, but which may be manifested by children enrolled in a regular 
school. The second part of the instrument, the Competence Scale, contains 
•items which focus on the child's mastery of the major areas of social compe- 
tence in. the preschool, including the quality and quantity of chi 1 d- tv- >r.her 
interactions, peer interactions, and autonomous involvement in activities. 

Items on this scale cover various degrees of both unhealthy as well as healthy 
functioning. 

In the previous study using the Kohn instrument (Stern, et al , 1972), a 
number of changes were made to be more responsive to conditions in the local 
Head Start programs. Further adaptations were made for the present inves- 
tigation. The Stern-Nummedal modification of the Kohn instrument excluded items 
which were redundant (i.e., the same concept presented in several ways), 
not behavioral ly-defined, or not applicable to the Head Start situation (for 
example, the average three-hour Head Start day does not usually include nap 
periods). Items were combined where H was felt that there were no behavioral 
distinctions to be made. Additional items were generated when a single 
existing item contained reference to behaviors which would not necessarily co- 
occur, or when behaviors deemed significant and applicable to the Head Start 
situation had not been included. With these excisions and additions the 32- 
item Behavior Checklist was increased to 38 items, and the Competence Scale was 
reduced from 75 to 54 items. The three-point rating scale for items on the 
Behavior Checklist was retained but the names given to the points on the 
scale were altered to read (1) Not at all typical, (2) Somewhat typical, and 
(3) Very typical. For the Competence Scale, the original seven point rating 
scale was collapsed into a five point scale: (1) Always, (2) Often, (3) Some- 
times, (4) Raraly, and (5) Never. Kohn's original summary Items for children's 
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overall functioning and verbal fluency were retained, but with a different 
format. The physical layout of the protocol was modified so as to facilitate 
the ease with which it could be used both by the ooservers and later by the 
data reduction r^aff. 

Kohn's factor analytic findings indicated the existence of two factors 
for each part of the instrument. For the Behavior Checklist, Factor 1 
consisted of items measuring apathy and withdrawal, and Factor 2 was made 
up of items measuring anger and defiance. For the Competence Scale, Factor 1 
contained items measuring interest-participation vs. apathy-withdrawal and 
Factor 2 consisted of items measuring cooperation-compliance vs. anger-defiance. 
The correlation between Checklist and Competence Scale for Factor 1 and 2 
were .75 and .79 respectively. 

In their work, Kohn and Rosman have relied completely on ratings provided 
by the children's own teachers, and this procedure was followed in the 
selection of disturbed children for the first therapeutic intervention study 
(Stem, et al , 1972). The teachers were asked to identify children for both the 
experimental treatment as well as the normal control group. Although the 
teacher's ratings were used as the basis for selection, observations were also 
made by the paraprofessional aides prior to inititation of the intervention and 
immediately after treatment had been terminated. 

The analysis of the data revealed that individual teachers had quite 
different baselines for determining the degree to which their children were de- 
parting from normalcy, and the tolerance limits they were willing to accept 
before assigning a child to the therapeutic treatment. Some of the children in 
the normal control group were described by the objective observers as demon- 
strating more aberrant behavior than children in the disturbed groups in another 
teacher's class. 

To avoid the problem of shifting baselines, and to establish a more 
accurate picture of the expected range of behaviors, a large scale study was 
designed to assess the extent of emotional disturbance among Head Start 
children and to provide a consistent basis for selection if therapeutic in- 
tervention were indicated. 



Method 



T he Study Population . A total of 413 children in 32 classrooms in 15 
individual Head Start sites under 4 delegate agency auspices were tested. . 
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Table 1 depicts the distribution by sex, race, and age for each agency. 

Agency 1 is represented by just 1 site and therefore has relatively fewer 
cases than the others. Agencies 2, 3, and 4 have 5, 4, and 4 sites each, 
respectively. In general there were an approximately equal number of boys 
and girls. The majority of the children were between 48 and 59 months of age, 
and the ethnic breakdown indicates that Blacks made up 80%, Mexican Americans 
13.8%, and Caucasian-Others 6.2% of the groups. This distribution is rep- 
resentative of the general Head Start population in the Los Angeles area. 

Procedure 

Training . Prior to embarking on the actual observation phase, a rigorous 
, training program was carried out with the observers. The individuals selected 

for this task all met the prerequisite of at least one year's experience 
testing and/or observing preschool children. Eight observers participated 
■ in a total of four half-day training sessions. On the first day an overview 

| of the project was provided, with general guidelines for on-site behavior. 

| The protocols for the Behavior Checklist and Competence Scale were then dis- 

tributed and each item discussed in detail, with both positive and negative 
instances of the types of behavior covered by the particular item. 

The second day gave the trainees the opportunity to apply the instrument 
to videotaped records of a variety of Head Start classroom situations. These 
tapes had been made during the therapy study already cited (Stern, et al . , 

1972) and provided excellent illustrations of both normal and disturbed behavior. 
For the third day, practice observation of children in Head Start classes had 
been arranged and the observers spent two of the four hours out in the field 
observing structured and free play situations. 

Then they returned to the office for further discussion and clarification. 

On the fourth day pairs of observers were assigned to filling out protocols 
on the same children. Only after achieving reliability were the observers 
allowed to begin rating the children in the study population. 

Observation . Each observer spent four days in a classroom of approximately 
15 children. The first day's observation was rather global, allowing the 
observer to learn the names of the children and become familiar with the 
c^ssroom records and schedules. During the second, third, and fourth days, 
children were observed individually, first in five minute intervals and then 




Table 1. Demographic Description of Study Sample. 





Agency 


Age 


Black 
M F 


Mexican 
African 
M F 


Caucasian- 
Other 
M F 


Total 
M F 


Total 



1 


3 


0 


0 


1 


0 


1 


0, 


2 


0 


2 


« 

ro 

CO 


4 


8 


6 


5 


6 


0 


1 


13 


13 


26 


5 


0 


0 


0 


0 


0 


0 


0 


0 


0 


2 


3 


16 


17 


0 


1 


0 


2 


16 


20 


36 


(N=159) 


4 


56 


52 


6 


1 


4 


2 


66 


55 


121 


5 


1 


1 


0 


0 


0 


0 


1 


1 


2 


3 


3 


15 


10 


1 


0 


0 


1 


16 


11 


27 


(N=106) 


4 


38 


39 


0 


0 


0 


0 


38 


39 


77 


5 


1 


1 


0 


0 


0 


0 


1 


1 


2 


4 


3 


6 


3 


4 


4 


4 


0 


14 


7 


21 


(N=l 20) 


4 


32 


26 


14 


13 


5 


6 


51 


45 


96 


5 


2 


0 


0 


1 


0 


0 


2 


1 


3 


Total 


3 


37 


30 


6 


5 


5 


3 


48 


38 


86 


( N=41 3) 


4 


134 


123 


25 


20 


9 


9 


168 


152 


320 


5 


4 


2 


0 


1 


0 


0 


4 


3 


7 


Total 




175 


155 


31 


26 


14 


12 


220 


193 


413 
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in groups. The frequency of occurrence of behaviors listed on the screening 
instrument were recorded on a special Daily Record Sheet . At the end of 

the fourth day of observation the Behavior Checklist and Competence Scale was 
completed for each child in the class. Teachers were also asked to fill out 
the instrument for the individual children. 



Reliability 

A major question in all observation studies is the degree of reliability 
of the data. In the present study two types of reliability were obtained: 1) 
Observer-Teacher and 2) Observer-Observer. For the first reliability estimate, 
the ratings of eight observers and 32 teachers of the 413 children on each of 
the variables as well as the overall ratings were compared. These are 
presented in Table 2. Although only moderate in size, all of the r^s are sig- 
nificant at above the. 01 level of significance. It should be noted that, as 



Table 2. 


Teacher-Observer Agreement on 


Rating 


Scales . 






B.C. 


B.C. 


C.S. 


— rri 


Total 


Total 


Overall - 

Ratings 




Factor 1 


2 


1 


2 


1 


2 






.53 


.60 


.35 


.48 


.43 


.52 


.29 





Mote; All correlations are significant at the p<.01 level. 

Number of Children Observed: 413; Number of Observers: 8; Number of Teachers: 32 



expected, the least reliable rating was that of the overall impression of 
the child's behavior. It is a curious fact that teachers who make many astute 
observations on specific behaviors will tend to give average or near-average 
ratings on a global basis. 

Tests of reliability of observation instruments are exceedingly vulnerable 
since it is difficult to insure that the two observers are attending to pre- 
cisely the same events at the same time. To maximize the precision of the 

^Ccoies of all instruments used in the study are presented in Appendix A. 
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reliability estimate, the observers in the present study were given the list 
of precise problem-behaviors for the specific children to be observed and 
asked to record the frequency of their occurrence within limited segments of 
time. Table 3 presents the data from this inter-observer reliability test. 

As can be seen from this table, they demonstrate uniformly and acceptably high 
correlations. 



Table 3. Estimates of Inter-Observer Reliability 



Observer Pairs 


1 


2 


3 


4 


5 


6 




Number of Children 
Observed 


19 


30 


35 


16 


58 


22 




r 


0.94** 


0.81** 


0.80** 


0.99** 


0.98** 


0.99** 





**p <.01 



Scoring the Behavior Checklist and Competence Scale . 

In identifying disturbed children from the data obtained with the obser- 
vation measures, Kohn (1968) has recommended the simplified procedure of 
pooling the corresponding factor scores from the Checklist and the. Competence 
Scale. However, the unequal number of items in the two scales, with fewer in 
the Checklist which is considered to be the more clinically oriented of the 
two, made the use of raw scores inappropriate. Thus the four raw score 
distributions were converted to Z scores J All the analyses presented are 
in terms of these standardized scores. 

Results 

In Table 4 are presented, by site, the means and standard deviations 
for both factors of the Behavior Checklist and the Competence Scale, as well 
as the total factor scores. To determine whether there were any significant 
differences on these variables as a function of Delegate Agency affiliation, 
a one-way analysis of variance was run separately for each of the Individual 
factors and for the combined factor scores. The results of this analysis, 
shown in Table 5, revealed no statistically significant differences among 
agencies on five of the six variables. However, for Factor 2 of the Checklist, 



'These conversion tables are presented in Appendix B. 



which measures the extent of clinically defined aggressive, acting-out be- 
havior, there were significant differences among the three agencies. The 
Newman-Keuls post hoc test confirmed that the frequency of this type of be- 
havior was significantly greater in Agency 4 compared to Agency 3, and in 
Agency 3 compared to Agency 2. 

The next analysis (see Table 6), comparing the 14 sites independent of 
agency, indicated significant differences on all but the Total Factor 1 
scores. Again, Newman-Keuls analyses were performed to identify the source 
of differences on each of the dependent measures. These tables are presented 
in detail in Appendix B. For the Competence Scale, only the sites at the two 
extremes of the distribution, both from different agencies, were significantly 
different from each other on Factor 1, with no other site showing meaningful 
differences on the Apathy- Withdrawal dimension. A similar finding for the 
corresponding factor on the Checklist showed only two sites at the upper 
extreme as significantly different from the one site at the lower end of the 
distribution. All three of these sites were in different agencies. 

There were more and larger between-site differences for Factor 2 on both 
the Checklist and the Competence Scale. However, on the Checklist these all 
related to site 4, which was significantly different from six other sites; 
three of these were from a single (different) agency, and one from the same 
agency. On the Competence Scale, there were Factor 2 differences involving 
four of the same six sites. These data indicate that the bases of the site 
differences could not be attributed to differences in the child development 
concepts or mental health orientation which a particular agency might favor. 



Table 4- 



Means and Standard Deviations for Factor 1 and Factor - 
Scorns (by Site). 



Si te 


N 




Factor 1 


Factor 2 


Factor 1 


Factor 2 


Factor 1 


Factor 2 




B.C. 


B.C. 


C.S. 


C.S. 


Total 


Total 


1 


28 


Mean 

S.D. 


-0.49 

1.47 


0.05 

1.15 


0.62 

1.29 


-0.38 
1 .01 


1.11 

2.63 


-0.33 

2.08 


2 


56 


Mean 

S.D. 


-0.21 

0.82 


-0.17 

0.67 


0.03 

1.10 


-0.54 

0.82 


-0.18 
1 .85 


-0.70 
1 .44 


3 


20 


Mean 

S.D. 


0.43 
1 .43 


-0.19 

0.89 


0.43 

1.34 


-0.54 

0.82 


0.85 

2.67 


-0.72 

1.61 


4 


28 


Mean 

S.D. 


-0.07 

0.80 


0.58 

1.48 


0.48 

0.92 


-0.21 

1.07 


0.41 
1 .57 


0.79 
2 . 00 . 


5 


30 


Mean 

S.D. 


0.50 

1.28 


0.38 

1.47 


0.27 

1.33 


C.09 

1.67 


0.76 

2.52 


0.48 

3.09 


6 


25 


Mean 

S.D. 


-0.03 
1 .22 


-0.48 

0.43 


-0.11 

1.35 


-0.94 

0.48 


-0.14 

2.54 


-i .41 
0.85 


7 


29 


Mean 

S.D. 


-0.47 

0.50 


-0.29 

0.84 


-0.16 

0.81 


-0.66 

0.80 


-0.62 

1.16 


-0.94 

•1.56 


8 


29 


Mean 

S.D. 


-0.04 

0.94 


0.00 

0.63 


-0.28 

0.95 


-0.53 

0.67 


-0.32 

1.81 


-0.52 

1.20 


9 


25 


Mean 

S.D. 


0.34 
1 .37 


0.00 

0.87 


0.46 
1 .20 


-6.03 

0.92 


0.79 

2.45 


-0.02 

1.70 


10 


23 


Mean 

S.D. 


0.18 

1.20 


-0.36 

0.47 


0.12 

0.94 


-0.44 

0.63 


0.30 

2.11 


-0.80 

1.05 


11 


50 


Mean 

S.D. 


-0.15 

0.71 


-0.42 

0.40 


0.08 

0.90 


-0.56 

0.70 


-0.07 
1 .47 


-0.97 

1.04 


12 


20 


Mean 

S.D. 


0.02 

0.82 


-0.46 

0.45 


0.16 

1.02 


-0.53 

0.69 


0.18 

1.73 


-0.98 

1.09 


13 


21 


Mean 

S.D. 


-0.13 

0.84 


-0.24 

1.13 


0.50 
1 .42 


-0.07 

1.21 


0.38 

2.14 


-0.31 

2.28 


14 


29 


Mean 

S.D. 


-0.24 

0.79 


0.00 

1.69 


0.51 
1 .33 


-0.27 
1 .04 


0.27 

2.04 


-0.27 

2.58 



Note — Site 1= Agency 1; Sites 2, 3, 4, 5, 6 = Agency 2; Sites 7,8,9,10 = Agency 3; 
Sites 11,12,13,14 = Agency 4. 
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Table 5. 



Analysis o* Variance for Factor 1 and Factor 2 Scores, with 
Agency as Source of Variance. 



Variable 


Agency 


Mean 


S.O. 


MS 


df 




2 


0.06 


1.10 






Factor 1 , B .C. 


3 


-0.02 


1 .06 


1.29 


2 


4 


-0.14 


0.77 








2 


0.01 


1.09 






Factor 2 , B.C, 


3 


-0.16 


0.73 


3.29 


2 




4 


-0.29 


1.01 








2 


0.18 


1.20 






Factor 1 , C.S. 


3 


0.01 


1 .01 


1.92 


2 




4 


0.27 


1.13 








2 


-0.35 


1 i.ii 






Factor 2 , C.S. 


3 


0.43 


0.79 


0.21 


2 




4 


-0.40 


0.90 








2 


0.24 


2.19 






Total Factor 1 


3 


-0.01 


1 .96 


1.88 


2 




4 


0.13 


1 .78 








2 


-0.33 


2.13 






Total Factor 2 


3 


-0.58 


1.43 


4.67 


2 




4 


-0.69 


1.79 







*p —.05 
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ERIC 



1.30 

3.44* 

1.51 

0.23 

0.47 

1.36 



12 



Table 6. Analysis of Variance for Factor 1 and Factor 2 Scores, 
with Site as Source of Variance. 



Variable 


df 






MS 




F 


Factor 1 , B.C. 


13 






2.58 




2.49* 


Factor 2, B.C. 


13 






2.77 




2.97* 


Factor 1 , C.S. 


13 






2.30 




1.80* 


Factor 2, C.S. 


13 






2.81 




3.23* 


Total Factor 1 


13 






7.25 




1.74 


Total Factor 2 


13 






10.13 




3.09* 


*p < . 05 














To answer 


the question of whether particular sites within the 


same agency 


differed from one another, additional 


analyses 


of variance 


were computed (see 


Table 7). It is interesting to 


note that there arc no significant 


wi thin-site 


Table 7, 


Analysis of Variance 


for Factor 1 and Factor 2 Scores by 




Site, Separately for Each Agency. 








Agency 2 (N=159) 


Agency 3 (N=106) 


Agency 4 (N=l 20 ) 


Variable 


MS df 


F 


MS 


df F 


MS 


df F 


Factor 1 , B.C. 


3.31 4 


2.89* 


3.28 


3 3.09* 


0.28 


3 0.47 


Factor 2, B.C. 


5.48 4 


5.10* 


0.93 


3 1.77 


1.30 


3 1 .28 


Factor 1 , C.S. 


1.81 4 


1.27 


2.85 


3 2.96* 


1.66 


3 1 .29 


Factor 2, C.S. 


6.48 4 


5.97* 


1.97 


3 3.37* 


1.44 


3 1 .80 


Total Factor 1 


7.52 4 


1.59 


10.61 


3 2.91* 


1.30 


3 0.41 


Total Factor 2 
AC 


23.73 4 


5.85* 


4.23 


3 2.31* 


4.63 


3 1.47 



differences at Agency 4. Looking at Table 1, it can be seen that this Agency 
had a far more heterogeneous ethnic mix than the others; it also had a far 
smaller percentage of young children: 17.5% compared to 21.5% for the total 

population. Variations among sites in the other two agencies do not seem 
to fellow any specific pattern. It is quite possible that the obtained 
differences are merely statistical artifacts, reflecting random population 



changes which most teachers have witnessed from class to class and year to year. 




Developing the Selection Criteria 

Having been satisfied that the observed children were fairly represen- 
tative of the range generally found in the local Head Start program, the final 
ohase of the study was devoted to establishing appropriate criteria for 
deciding which children were in need of therapeutic intervention. Because the 
Behavior Checklist was developed to measure the frequency of aberrant behavior, 
the distribution was heavily skewed in the positive direction; that is, the 
majority of the children were clustering in the normal range, with only a small 
proportion demonstrating highly deviant scores. On the other hand, the Com- 
petence Scale, which was designed to assess normal functioning, was expected 
to yield an approximately normal distribution curve. Finally, the total score, 
which combined corresponding factors from the two scales, was expected to 
yield a modified normal distribution with a slightly positive skew. The ob- 
tained data followed the anticipated pattern. Table 8 presents the percent 
of the population falling in the interval whose upper limit is given 
in the Z score columns. 

A similar distribution for Z scores obtained with the Competence Scale is 
not included for several reasons. Two alternatives for establishing criteria 
for intervention seemed defensible. First, children could be selected on the 
basis of scores on the Behavior Checklist alone. This procedure has been 
recommended by Kohn, since the measure was designed to detect overt symptoms 
of deviant behavior. The second alternative, also suggested by Kohn, was to 
combine scores on the two measures. The Z score transformations made it 
possible to obtain a simple set of scores in which the items of the Checklist 
would be given appropriate weight. A selection criteria based on the Com- 
petence Scale scores alone would be demonstrably inadequate; hence there seemed 
to be no reason for including a Z score distribution for the Competence Scale. 

The major consideration in the selection problem is that of establishing 
the cut-off score. According to the theory of normally-distributed attributes, 
cn a two-tailed test a Z score of 1.96 or above would identify atypical or 
ceviant children with a 5% level of error. That is, there would be a chance 
that 5 out of 100 children might be erroneously identified as disturbed. A 
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Table 8. Distribution of Scores and Cut-Off Criteria for Selection 
of Subjects. (N=413) 



Behavior Checklist 




Total 


Factor Scores (BC + 


CS) 


Factor 
Z- Score 


1 

% 


Factor 

Z-Score 


9 

u 

o / 
/r> 


Factor 

Z-Score 


1 


Factor 

7-Score 


n 

L. 

»v 


-0.83 


22.8 


-0.67 


48.5 


-1 .^8 


27.6 


-0.96 


59.6 


-0.68 


14.0 


-0.47 


12.3 


-1.22 


6.1 


-0.61 


10.9 


-0.53 


10.4 


-0.27 


8.0 


-0.96 


6.5 


-0.27 


6.5 


-0.38 


9.0 


-0.07 


5.8 


-0.70 


6.0 


0.1 A 


5.3 


-0.23 


4.9 


0.13 


4.8 


-0.44 


6.8 


0.48 


2.0 


-0.08 


5.1 


0.33 


3.9 


-0.18 


4.9 


0.83 


1 .9 


0.07 


4.1 


0.53 


2.4 


0.08 


5.3 


1 .17 


3.4 


0.22 


4.6 


0.73 


2.7 


0.35 


3.4 


1 .52 


1 .2 


0.37 


4.1 


0.93 


1.9 


0.61 


4.6 


1 .86 


1 .0 


0.51 


0.0 


1 .13 


1.0 


0.87 


2.9 


2.20 


0.7 


0.66 


2.9 


1.33 


1.7 


1 .13 


2.4 


2.54 


1 .2 


0.81 


0.7 


1 .53 


0.7 


1.39 


1.9 


2.89 


0.5 


0.96 


2.2 


1 .73 


0.2 


1 .65 


3.4 


3.23 


0.7 


1 .11 


1 .9 


1 .93 


0.5 


1 .91 


1.5 


3.51 


0.5 


1.26 


1 .7 


2.13 


0.7 


2.17 


2.4 


3.92 


0.5 


1 .41 


1 .2 


2.33 


0.7 


2.43 


1.5 


4.26 


9.2 


1 .56 


1 .7 


2.53 


0.5 


2.69 


1.2 


4.61 


0.2 


1.71 


0.2 


2.73 


0.5 


2.96 


1 .2 


4.95 


0.9 


1 .86 


0.0 


2.93 


1.0 


3.22 


0.7 


5.29 


0.2 


2.00 


1 .2 


3.13 


0.2 


3.48 


1 .0 


5.64 


0.9 


2.15 


2.2 


3.33 


0." 


3.74 


1.2 


5.98 


1 .5 


2.30 


0.7 


3.53 


1 .0 


4.00 


2.2 






2.45 


0.5 


3.73 


0.5 


4.26 


5.3 






2.60 


0.7 


3.93 


* 










2.75 


0.5 


*7 












2.90 


0.2 


4.73 


0.3 










3.05 


0.7 




4 

/ 










3.20 


0.2 


6.93 


0.2 










3.35 


0.2 














3.49 


0.2 














3.64 


0.2 














3.79 


0.2 














3.94 


0.2 














4.09 


0.2 














4.24 


0.2 














4.39 


0.2 














4.54 


0.2 
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more rigorous criterion, for example a Z score of 2.58, at the 1% level, would 
reduce that probability so that only 1 out of 100 children might be incorrectly 
identified. For the Behavior Checklist, in which aberrancy is computed in 
only one direction from the mean, a one-tailed test might be considered 
more appropriate. Thus Z scores of 1.64 and 2.33 would be at the 5% and 1? 
level of probability, respectively. 

It should be remembered, however, that as the chances for selecting 
inappropriate cases (Type I error) decrease the chances for over-looking 
appropriate cases (Type II error) increase. In the final analysis, the 
choice between these two types of error becomes as much a matter of the per- 
sonality of the investigator as that of the sensitivity of the particular 
experiment. The probability of error can be reduced if more than one judgment 
point can be obtained. Hence the alternative of summing the Z scores from 
the Checklist and Competence Scale for the corresponding factor would offer 
some increased confidence in the accuracy of the diagnosis. If the Competence 
Scale is considered a two-tailed test and the Behavior Checklist a one-tailed 
test, the Z scores at the 5% levels of both tests would provide a combined 
Z score of 3.60 as the cut-off point. On Table 8 it can be seen that a 
criterion at the 5% Tevel based on the Checklist alon? would characterize 
14.3% of the study population as disturbed; with the combined score as cut-off 
criterion only 13.1% are so identified. If one remembers that Head Start 
is not compulsory and that most of the severely disturbed preschool age children 
are not accepted for enrollment, the percentage of disturbed children in the 
study population seems to be slightly higher than the figure cited for 
elementary school children in the Joint Commission reports. Evidently many 
of these children fall out to become part of the close to a million and a half 
children characterized as suffering from "severe mental problems." 

The present study, in addition to the work of Kohn & Rosman, seems to 

support the usefulness of this instrument and procedure for identifying, at an early- 
age, children who are likely to become much more severe problems later on. 

The question of whether therapeutic intervention with this population can have 
an appreciable Impact on changing this prediction was the subject of the 
subsequent study. 
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UCLA EARLY CHILDHOOD RESEARCH CENTER 
1970-1971 



EXAMINER'S MANUAL 
FOR 

BEHAVIOR CHECKLIST AND COMPETENCE SCALE 



The Behavior Checklist and the Competence Scale have been designed to 
measure problem behaviors as well as the degree of mastery the three- to 
five-year-old child demonstrates in the Head Start environment. While it 
should be borne in mind that all children exhibit some of the behaviors 
at some point in their development, the items are to be rated in terms of 
the frequency with which they have been observed during the period In 
which the child has been in the Head Start class. 

The Rehavior Checklist consists of 38 behavior items which are to be 
rated on a three-point scale: 0 = Not at all typical; 1 « Somewhat 
typical; and 2 = Very typical. 

The ComDetence Scale includes 54 items, each of which is to be rated: 
0=Always; l=0ften; 2=Sometimes; 3=Rarely; and 4=Never. 

Please be careful that the response checked corresponds with the 
appropriately numbered item. Items 55-57 are summary evaluations. 



RATING INSTRUCTIONS 

1. Consider each question independently. It is well known that children 
may exhibit seemingly contradictory behavior. 

2. Base your ratings on how the child functions in Head Start. 

3. Some items contain a number of specific behaviors which are only 

slightly different from each other. Do not hesitate to make a rating 

even though the child does not exhibit all of the specific behaviors. 

4. Answer every item. Do not leave any blanks. 

5. Do not hesitate to use the extreme points where appropriate. 



The Behavior Checklist and the Competence Scale have been adapted 
from Martin Kohn, Competence and symptom factors in the preschool child, 
William A. White Institute, New York, 1968, by the Head Start Research 
Center, University of California, Los Angeles, September 1969. 
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BEHAVIOR CHECKLIST 



1. Eats, sucks or drinks inedible substances such as plaster, ink, sand, 
cloth, wood, toys, etc. 

2. Body appears stiff and rigid when standing sitting, lying down or 
being held. 

3. Has mournful, downcast expression, seams sad. 

4. Is listless or apathetic. 

5. Moves head or body in a very slow way. 

6. Hardly speaks at all. 

7. Speaks in a faint or weak voice (not due to known physical causes). 

8. Acts overly fearful and cautious.. 

9. Acts extremely frightened to the point of screaming and crying (dis- 
regard where child screams or cries out of anger). 

10. Screams or bangs objects, etc., when angry, irritated or frustrated. 
(Has temper tantrums). 

11. Gets easily irritated or bothered by things or by people. 

12. Is tense or jittery in everyday situations or activities. 

13. Gets angry when Interrupted at play by adult as part of normal 
routine (not punishment). 

14. Seeks attention through rowdy or "show-off" behavior. 

15. Withdraws or accepts it (does not defend himself) when others shove, 
hit, accuse or criticize him. 

16. Stays close or clings to mother or adult (more than usual for this 
age level). 

17. When working with materials, becomes bothered or upset when he cannot 
make things "just right" or put them in perfect order. 

18. Says he's going to kill himself. 

19. Talks about death and killing. 

20. "Talks back" to adults in a defiant and fresh manner. 
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21. Is unresponsive to or resents friendly overtures by adults or children. 

22. Maltreats other children, with deliberate cruelty; bullies, hits or 
picks on other children. 

23. Gives excuses for not following classroom procedures or rules of game. 

24. Hurts others, tells untruths, or destroys property without seeming to 
feel badly. 

25. Is very much a follower; does not initiate play with other children 
(fails to start or suggest gamer* or to Initiate activities). 

26. Swears or curses, using "hell," "damn," or other four-letter words . 

27. Occupies himself continuously with one type of activity and resists 
leaving It to do anything else. 

28. Has an aloof and distant manner (keeps to himself). 

29. Requires urging to take part In activities. 

30. Is shy or bashful and falls to play with most other children. 

31. Stares blankly Into space without appearing to be looking at or think- 
ing about anything (to the point of being unresponsive to things or 

to people). 

32. Appears bewildered or confused. 

33. Is destructive in regard to his own and/or other's property. 

34. Attempts to draw attention to himself when teacher pays attention to 
other children. 

35. Is overly sensitive; feelings easily hurt. 

36. Is shy or bashful with most adults. 

37. Often starts fights with other children (physical or verbal). 

38. Disobeys directions or Instructions of adult. 
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COMPETENCE SCALE 

Child-Teacher 

1. Child rebels physically, e.g., has temper tantrums, hits, kicks, etc. 

2. Child frowns, shrugs shoulders, pouts or stamps foot when suggestion 
is made by teacher. 

3. Child responds well only if the activity Is planned or directed by 
the teacher. 

4. Child accepts teacher's ideas and suggestions for play or ways of 
playing. 

5. Child reacts negatively to teacher's ideas and suggestions for play 
activities. 

6. Child dawdles when required to do something. 

7. Child cooperates with rules and regulations. 

8. Child hits teacher. 

9. Child refuses to carry out reasonable suggestions from teacher even 
when having difficulty. 

10. Child responds with immediate compliance to teacher's direction. 

11. Child carries out requests and directions. 

12. Child needs adult aid for each step of activity. 

13. Child expresses open defiance against teacher's rules and regulations. 

14. Child is independent of adult in planning his activities. 

15. Child is independent of adult in overcoming difficulties with other 

children or activities. 

16. Child is able to make his needs known to the teacher. 

17. Child adds freely to suggestions made by the teacher. 

Child-Child 

18. Child Is open to the Ideas and suggestions of other children. 

19. Child refuses to participate in activities with other children unless 

he can be the leader. 

2C. Child resists going along with the ideas of other children. 
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21. Child disrupts activities of others. 

22. Child is fearful in approaching other children. 

23. Child's ideas have little impact on many children In the classroom. 

24. Child is at a loss without other children directing activities for 
him. 

25. Child takes possession of another child's materials without per- 
mission. 

26. Child is unwilling to play with other children except on his own terms. 

27. Child is bossy and dominating with other children. 

28. Child quarrels with other children. 

29. Child gets others interested in what he Is doing. 

30. Child is hostile or aggressive with other children. 

31. Child shies away and withdraws when approached by other children. 

32. Child is bossed and dominated by other children. 

33. Child has difficulty defending his own rights with other children. 

34. Child attempts to gain cooperation from other children through 
threats. 

35. Child prefers aggressive types of play. 



Child Activities 

36. Child acts silly at lunch time, e.g., giggles, shrieks, etc. 

37. Child demonstrates little interest In things and activities. 

38. Child manifests interest in many and varied types of things. 

39. Child has difficulty in getting the attention of the group. 

40. Child easily loses interest and flits from one activity to another. 

41. Child is destructive with materials. 

42. Child becomes disruptive when frustrated in an activity. 

43. Child seems to be at a loss when first coming into the classroom in 
the morning. 
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44. Child prefers to play without any restrictions. 

45. Child*engages in activities in which there is a high probability of 
hurting himself. 



46. 

47. 

48. 

49. 

50. 



Child resists making a change from one activity to a new activity. 
Child gets into trouble in unstructured free-play types of activities. 
Child puts things away carefully. 

Child throws food and messes himself v/hen eating. 

Child has trouble keeping to the rules of the' game. 




51. Child seems eager to try new things. 

52. Child spends time sitting around, looking around, or wandering around 
aimlessly. 

53. Child is able to follow routines, e.g., putting on and taking off own 
sweater or jacket, washing hands. V 

54. Child participates in a half-hearted v/ay. 



Summary Items 



55. Overall functioning : 1 * Good; 2 ° Moderately good; 3 = Poor. 

1. Good or well functioning . Gets along well with teachers and other 
children; participates with interest In activities, accomplishes 
tasks usually mastered by children his age; alert, self- 
sufficient, friendly, and unafraid. 

2. Moderately well -functioning . Functions fairly adequately but 
shows some difficulty in one or several areas; growing up repre- 
sents something of a struggle; not as happy as he might be. 

3. Poorly functioning . In comparison with other in the group pre- 
sents problems and exhibits signs of disturbance, e.g., getting 
along with teachers and adults (over-demanding of the teacher's 
attention; or completely rejecting her); relating to other chil- 
dren (bossy and antagonistic, or very withdrawn and frightened); 
inability to sustain any prolonged Interest. 

56. Verbal fluency, ability in self-expression : 1 =* Superior; 2 = Average; 

3 * Poor; 4 = Minimal, or no speech. 

57. L anguage used by child in class : 1 = English only; 2 ■ Primarily 

English with occasional use of a second language; 3 = Uses both 
languages equally; 4 = Primarily another language, only minimal 
English; 5 = Not appropriate. Child has no or only minimal speech 
of any ;ind. 
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Z Score Conversion Table for Behavior Checklist. 

Table 2. Z Score Conversion Table for Competence Scale. 

Table 3. Z Score Conversion Table for Total Factor 1 & 2. 
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for Newman-Keuls Post Hoc Test, Indicating Level of Signi- 
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Table 1. Z Score Conversion Table for 8ehavior Checklist. 



Factor 1 


Factor 2 


Mean 5.55; S.D. G.71 
Range of Raw Scores: 0—36 

Range of Z Scores: -.83— <-4.84 


Mean 3.37; S.D. 5.00 
Range of Raw Scores: 0-38 

Range of Z Scores: -.67—4-6.93 



Raw 

Score 


Z Score 
FI F2 


Raw 

Score 


7 . Score 
FI FP. 


0 


- . 83 


-.67 


20 


2.15 


3.33 
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-.68 


-.47 . 


21 


2.30 


3.53 
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-.53 


-.27 


22 


2.45 


3.73 
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-.38 


-.07 


23 
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3.93 
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.13 


24 
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4.13 


5 


-.08 


.33 


25 


2.90 


4.33 


6 


.07 


.53 


26 


3.05 


4.53 


7 


.22 


.73 


27 


3.20 


4.73 


8 


.37 


.93 


28 


3.35 


4.93 


9 


.51 


1.13 


29 


3.49 


5.13 


10 


.66 


1 .33 


30 


3.64 


5.33 


11 


.81 


1.53 


31 


3.79 


5.53 


12 


.96 


1 .73 


32 


3.94 


5.73 


13 


1.11 


1.93 


33 


4.09 


5.93 


14 


1 .26 


2.13 


34 


4.24 


6.13 


15 


1 .41 


2.33 


35 


4.39 


6.33 


16 


1 .56 


2.53 


36 


4.54 


6.53 


17 


1 .71 


2.73 


37 


4.69 


6.73 


18 


1 .86 


2.93 


38 


4.84 


6.93 


19 


2.00 


3.13 
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Table 2a. Z Score Conversion Table for Competence Scale. Factor 1 



Mean 22.83 Ranqe of Raw Scores: 0-76 

S.D. 13.42 Ranqe of 7. Scores: -1.70 - +3.06 



Raw 

Score 


Z 

Score 


Raw 

Score 


Z 

Score 


Raw 

Score 


Z 

Score 


Raw 

Score 


7 

Score 


0 


-1.70 


20 


- .21 


40 


1 .28 


60 


2.77 


1 


-1.63 


21 


- .14 


41 


1.35 


61 


2.84 


2 


-1.55 


22 


- .06 


42 


1 .43 


62 


2.92 


3 


-1 .48 


23 


.01 


43 


1 .50 


63 


2.99 


4 


-1 .40 


24 


.09 


44 


1.58 


64 


3.07 


5 


-1.33 


25 


.16 


45 


1 .65 


65 


3.14 


6 


-1 .25 


26 


.24 


46 


1 .73 


66 


3.22 


7 


-1.18 


27 


.31 


47 


1 .80 


67 


3.29 


8 


-1.11 


28 


.39 


48 


1 .86 


68 


3.37 


9 


-1 .03 


29 


.46 


49 


1 .95 


69 


3.44 


10 


- .96 


30 


.53 


50 


2.02 


70 


3.51 


11 


- .88 


31 


.61 


51 


2.10 


71 


3.59 


12 


- .81 


32 


.68 


52 


2.17 


72 


3.66 


13 


- .73 


33 


.76 


53 


2.25 


73 


3.74 


14 


- .66 


34 


.83 


54 


2.32 


74 


3.81 


15 


- .58 


35 


.91 


55 


2.40 


75 ' 


3.89 


16 


- .51 


36 


.98 


56 


2.47 


76 


3.96 


17 


- .43 


37 


1.06 


57 


2.55 






18 . 


- .36 


38 


1.13 


58 


2.62 






19 


- .29 


39 


1.20 


59 


2.70 
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Table 2b. Z Score Conversion Table for Competence Scale, Factor 2. 

Mean 10.58 Range of Raw Scores: 0-68 

S.D. 10.20 Range of 7. Score: -1.04 - +5.63 



Raw 

Score 


Z 

Score 


Raw 

Score 


Z 

Score 


Raw 

Score 


Z 

Score 


Raw 

Score 


Z 

Score 


0 


-1.04 


18 


.73 


36 


2.49 


54 


4.26 


1 


- .94 


19 


.83 


37 


2.59 


55 


4.35 


2 


- .84 


20 


.92 


38 


2.69 


56 


4.45 


3 


- .74 


21 


1 .02 


39 


2.79 


57 


4.55 


4 


- .65 


22 


1.12 


40 


2.88 


58 


4.65 


5 


- .55 


23 


1 .22 


41 


2.98 


59 


4.75 


6 


- .45 


24 


1 .32 


42 


3.08 


60 


4.85 


7 


- .35 


25 


1 .41 


43 


3.18 


61 


4.94 


8 


- .25 


26 


1 .51 


44 


3.28 


62 


5.04 


9 


- .15 


27 


1 .61 


45 


3.37 


63 


5 5.14 


10 


- .06 


28 


1.71 


46 


3.47 


64 


5.24 


n 


.04 


29 


1 .81 


47 


3.57 


65 


5.34 


12 


.14 


30 


1 .90 


48 


3.67 


66 


5.43 


13 


.24 


31 


2.00 


49 


3.77 


67 


5.53 


14 


.34 


32 


2.10 


50 


3.86 


68 


5.63 


15 


.43 


33 


2.20 


51 


3.96 






16 


.53 


34 


2.30 


52 


4.06 






17 


.63 


35 


2.39 


53 


4.16 







Table 3a. Z Score Conversion Table for Total Factor 1. 



t 

t 




fie an 28,38 
S.D. 19.16 








Range 

Range 


of Raw Scores 
of Z Scores: 


: 0-114 
-1 .48 - 


4.47 


Raw 

Score 


Z 

Score 


Raw 

Score 


Z 

Score 


Raw 

Score 


Z 

Score 


Faw 

Score 


7. 

Score 


0 


-1.48 


29 


.03 


58 


1 .55 


87 


3.06 


1 


-1.43 


30 


.08 


59 


1.60 


38 


3.11 


2 


-1.38 


31 


.14 


60 


1 .65 


89 


3.16 


3 


-1.32 


32 


.19 


61 


1.70 


90 


3.22 


4 


-1.27 


33 


.24 


62 


1 .75 


91 


3.27 


5 


-1 .22 


34 


.29 


63 


1 .81 


92 


3.32 


6 


-1.17 


35 


.35 


64 


1 .86 


93 


3.37 


7 


-1.12 


36 


.40 


65 


1 .91 


94 


3.42 


8 


-1 .06 


37 


.45 


66 


1 .96 


95 


3.48 


9 


-1 .01 


38 


.50 


67 


2.02 


96 


3.53 


10 


- .96 


39 


.55 


68 


2.07 


97 


3.58 


11 


- .91 


40 


.61 


69 


2.12 


98 


3.63 


12 


- .85 


41 


.66 


70 


2.17 


99 


3.68 


13 


- .80 


42 


.71 


71 


2.22 


100 


3.74 


14 


• - .75 


43 


.76 


72 


2.28 


101 


3.79 


15 


- .70 


44 


.82 


73 


2.33 


102 


3.84 


16 


- .65 


45 


.87 


74 


2.38 


103 


3.89 


17 


- .59 


46 


.92 


75 


2.43 


104 


3.95 


18 


- .54 


47 


.97 


76 


2.49 


105 


4.00 


19 


- .49 


48 


1 .02 


77 


2.54 


106 


4.05 


20 


- .44 


49 


1.08 


78 


2.59 


107 


4.10 


21 


- .39 


50 


1.13 


79 


2.64 


108 


4.16 


22 


- .33 


51 


1.18 


80 


2.69 


109 


4.21 


23 


- .28 


52 


1.23 


81 


2.75 


no 


4.26 


24 


- .23 


53 


1.28 


82 


2.80 


111 


4.31 


25 


- .18 


54 


1.34 


83 


2.85 


112 


4.36 


26 


- .12 


55 


1 .39 


84 


2.90 


113 


4.42 


27 


- .07 


56 


1.44 


85 


2.96 


114 


4.47 


28 


- .02 


57 


1 .49 


86 


3.01 
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Table 3b. Z Score Conversion Table for Total Factor 2. 



Mean 13.94 
S.D. 14.56 






Range of P.aw Scores: 0- 
Ranqe of Z Scores: -96 


106 

- +6.32 




Raw 


Z 


Raw 


7. 


Raw 


Z 


Raw 


7 


Score 


Score 


Score 


Score 


Score 


Score 


Score 


Score 


0 


^T703 


27 


.90 


54 


2.75 


81 


4.61 


1 


- .96 


28 


.97 


55 


2.82 


82 


4.67 


2 


- .88 


29 


1.03 


56 


2.89 


83 


4.74 


3 


- .82 


30 


1.10 


57 


2.95 


84 


4.81 


4 


- .75 


31 


1 .17 


58 


3.03 


85 


4.88 


5 


- .68 


32 


1.24 


59 


3.09 


86 


4.95 


6 


- .61 


33 


1.31 


60 


3.16 


07 


5.02 


7 


- .54 


34 


1 .38 


61 


3.23 


88 


5.09 


8 


- .48 


35 


1.45 


62 


3.30 


89 


5.16 


9 


- .41 


36 


1.52 


63 


3.37 


90 


5.22 


10 


- .34 


37 


1.58 


64 


3.44 


91 


5.29 


11 


- .27 


38 


1.65 


65 


3.51 


92 


5.36 


12 


- .20 


39 


1.72 


66 


3.58 


93 


5.43 


13 


- .13 


40 


1.79 


67 


3.64 


94 


5.50 


14 


.00 


41 


1.86 


68 


3.71 


95 


5.57 


15 


.07 


42 


1.93 


69 


3.78 


96 


5.64 


16 


.14 


43 


2.00 


70 


3.85 


97 


5.70 


17 


.21 


44 


2.06 


71 


3.92 


98 


5.77 


18 


.28 


45 


2.13 


72 


3.99 


99 


5.84 


19 


.35 


46 


2.20 


73 


4.06 


100 


5.91 


20 


.42 


47 


2.27 


74 


4.12 


101 


5.98 


21 


.48 


48 


2.34 


75 


4.19 


102 


6.05 


22 


.55 


49 


2.41 


76 


4.26 


103 


6.12 


23 


.62 


50 


2.48 


77 


4.33 


104 


6.19 


24 


.69 


51 


2.54 


78 


4.40 


105 


6.25 


25 


.76 


52 


2.61 


79 


4.47 


106 


6.32 


26 


.83 


53 


2.68 


80 


4.54 
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Table 4a. Ordered Means and Critical Values, with Matrixes of Differences between Means and Studentized 
t-S tatis tics, for iiewman-Keuls Post Hoc Test, Indicating Level of Significance of Differences 
Between Adjacent Groups. Factor 1, Behavior Checklist. 
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'lote: All scores multiplied bv 100 to increase precision of interpretation. 



Table 4b. Ordered Means and Critical Value, with Matrixes of Differences Between Means and Student! zed 
t-Statistics, for Newman-Keuls Post Hoc Test, Indicatinq Level of Significance of Differences 
Between Adjacent Groups. Factor 1, Competence Scale. 
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Mote: All scores multiplied by 100 to increase precision of interoretation . 
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Note: All scores multiplied by 100 to increase orecision of interpretation. 



Table 4d. Ordered Means and Critical Values, with Matrixes of Differences Between Means and Studentizcd 
t-Statistics, for Newman-Keuls Post Hoc Test, Indicating Level of Significance of Differences 
Between Adjacent Groups. Factor 2, Competence Scale. 
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